Added pre-upgrade validation for N9K-C9408 with more than 6 N9K-X9400-16W LEM's for the bug CSCws82819 by Harinadh-Saladi · Pull Request #354 · datacenter/ACI-Pre-Upgrade-Validation-Script

Harinadh-Saladi · 2026-02-25T13:50:53Z

Added check for the bug CSCws82819 which has switch model N9K-C9408 with more than 6 N9K-X9400-16W LEM's in aci-preupgrade-validation-script.py
Added 11 test cases including Pass, Fail and NA cases.
Added documentation about the check and also about recommended action.

Tested in the lab and enclosed PASS, FAIL and NA scenarios.

CSCws82819_Logs.txt
CSCwn62369_Pytest_Logs.txt

Note: FAIL and PASS logs enclosed in the attachment are simulated logs based on the available switch and line card model as we don't have the models N9K-C9408 and N9K-X9400-16W in our lab.

dhaselva · 2026-03-03T09:19:30Z

+    headers = ["Node ID", "Switch Model", "LEM Model", "LEM Count"]
+    data = []
+    recommended_action = (
+        "Upgrade from pre-16.1(2f) to 16.1(2f) or later on N9K-C9408 with 6 or more LEMs will result in boot loop. Do NOT proceed. Use fewer LEMs or choose a different version.  "


Modify the recommended action message

Harinadh-Saladi · 2026-03-09T08:03:38Z

Enclosing the logs after testing with the changes.
CSCws82819_Logs.txt
CSCws82819_Pytest_Logs.txt

…2f and later with 6 or more LEMs'

… name for impacted nodes.Also, updated pytest test cases

…ith latest changes

Removed unwanted spaces

Updated validations.md file by modifying the impact and recommendation message.

Removed single quotes in the description.

Harinadh-Saladi · 2026-03-11T10:25:13Z

Enclosing the updated logs.

CSCws82819_Logs.txt
CSCws82819_Pytest_Logs.txt

lovkeshsharma702

please completed all comments.

Updated recommended action.

Updated workaround message.

…h-Saladi/ACI-Pre-Upgrade-Validation-Script into CSCws82819-c9408-boot-loop-check

takishida · 2026-04-03T07:35:27Z

Example result outputs:

[Check 1/1] N9K-C9408 with more than 5 N9K-X9400-16W LEMs...                                                       FAIL - OUTAGE WARNING!!
  Node ID  Switch Model  LEM Model      LEM Count
  -------  ------------  ---------      ---------
  101      N9K-C9408     N9K-X9400-16W  6
  102      N9K-C9408     N9K-X9400-16W  6

  Recommended Action: N9K-C9408 switches configured with >5 N9K-X9400-16W LEMs will enter a boot loop if upgraded to impacted release of CSCws82819. Upgrade to Fix version or Use less than 6 LEMS on impacted release
  Reference Document: https://datacenter.github.io/ACI-Pre-Upgrade-Validation-Script/validations/#n9k-c9408-with-more-than-5-n9k-x9400-16w-lems

* Add arg to define thread limit - to throttle concurrent API calls when required (#355) * add `--max-threads` arg * fix bad descriptor errs/race conditions * update pytests * 353 the script incorrectly detects vpc and port channel interfaces as cscwh68103 invalid fabricpathep targets (#357) * specific testing for known failure conditions of cscwh68103 as to not catch valid scenarios * Update aci-preupgrade-validation-script.py mark version * print cleanup * fix: Add cversion check for post_upgrade_cb_check (#377) * added validation for CFD CSCwp64296 (#307) Added rogue ep/coop exception mac check for the CFD CSCwp64296 * Added pre-upgrade validation for N9K-C9408 with more than 6 N9K-X9400-16W LEM's for the bug CSCws82819 (#354) * Added a new check for the bug 'CSCws82819N9K-C9408 boot loop on 16.1.2f and later with 6 or more LEMs' * New Validation for APIC Storage Inode Usage (F4388, F4389, F4390 equipment-full) (#361) * New Validation for APIC Storage Inode Usage (F4388, F4389, F4390 equipment-full) * Add new exception handling of invalid query filter in `icurl` due to, for example, a non-supported fault code on older versions * Add validation for multipod_modular_spine_bootscript_check - CSCwr66848 (#365) * Add check for CFD - CSCwr66848 * Update pytest.yml to run on vX.Y.Z branches * adding of cli parameters for user and password (#335) * added cli parameter for user (-u) and password (-p) to be used in an eased way for fully automated execution of 'aci-preupgrade-validation-script.py' * test: Add pytest --------- Co-authored-by: Detlef Sass <detlef.sass@de.bosch.com> * Added validation for CSCwd40071 (#332) * Added validation for CSCwd40071 * Addressed the comments * Added cversion for the check * Removed empty spaces * logic change. removed 0.0.0.0/0 and made pytest changes * logic modified and validation.md file updated * Added validation for CSCws84232 (#334) * Feat:added svccoreCtrlr excessive entries check * updated the formated json in svccore_negative.json * Removed other PR checks * Fix: svccoreNode mo check is updated * Fix:update the threshold to 240 * refactor:moved to general check * refactor:updated recommended action * refactor:optimise the check for get the count alone * refactor:added try catch exception for error handling * update the threshold value for 240 * refactor:query-target was added * refactor: removed unwanted spaces * refactor:removed unwanted spaces in validation.md * resolve conflict while rebasing * resolve conflict while rebasing * updated pytest script file * updated validation.md * addressed review comments * updated validation.md --------- Co-authored-by: takishida <38262981+takishida@users.noreply.github.com> * set to v4.1.0 --------- Co-authored-by: tkishida <tkishida@cisco.com> Co-authored-by: takishida <38262981+takishida@users.noreply.github.com> Co-authored-by: psureshb <psureshb@cisco.com> Co-authored-by: Harinadh-Saladi <hsaladi@cisco.com> Co-authored-by: sanjanch <sanjanch@cisco.com> Co-authored-by: asraf-khan <anazar@cisco.com> Co-authored-by: Thatleft <detlef.sass@web.de> Co-authored-by: Detlef Sass <detlef.sass@de.bosch.com> Co-authored-by: veesenth_cisco <veesenth@cisco.com>

* Add arg to define thread limit - to throttle concurrent API calls when required (#355) * add `--max-threads` arg * fix bad descriptor errs/race conditions * update pytests * 353 the script incorrectly detects vpc and port channel interfaces as cscwh68103 invalid fabricpathep targets (#357) * specific testing for known failure conditions of cscwh68103 as to not catch valid scenarios * Update aci-preupgrade-validation-script.py mark version * print cleanup * Unsupported hardware check * fix: Add cversion check for post_upgrade_cb_check (#377) * added validation for CFD CSCwp64296 (#307) Added rogue ep/coop exception mac check for the CFD CSCwp64296 * Added pre-upgrade validation for N9K-C9408 with more than 6 N9K-X9400-16W LEM's for the bug CSCws82819 (#354) * Added a new check for the bug 'CSCws82819N9K-C9408 boot loop on 16.1.2f and later with 6 or more LEMs' * New Validation for APIC Storage Inode Usage (F4388, F4389, F4390 equipment-full) (#361) * New Validation for APIC Storage Inode Usage (F4388, F4389, F4390 equipment-full) * Add new exception handling of invalid query filter in `icurl` due to, for example, a non-supported fault code on older versions * Add validation for multipod_modular_spine_bootscript_check - CSCwr66848 (#365) * Add check for CFD - CSCwr66848 * Update pytest.yml to run on vX.Y.Z branches * adding of cli parameters for user and password (#335) * added cli parameter for user (-u) and password (-p) to be used in an eased way for fully automated execution of 'aci-preupgrade-validation-script.py' * test: Add pytest --------- Co-authored-by: Detlef Sass <detlef.sass@de.bosch.com> * Addressed the review comments * Added validation for CSCwd40071 (#332) * Added validation for CSCwd40071 * Addressed the comments * Added cversion for the check * Removed empty spaces * logic change. removed 0.0.0.0/0 and made pytest changes * logic modified and validation.md file updated * Addressed the review comments * Added validation for CSCws84232 (#334) * Feat:added svccoreCtrlr excessive entries check * updated the formated json in svccore_negative.json * Removed other PR checks * Fix: svccoreNode mo check is updated * Fix:update the threshold to 240 * refactor:moved to general check * refactor:updated recommended action * refactor:optimise the check for get the count alone * refactor:added try catch exception for error handling * update the threshold value for 240 * refactor:query-target was added * refactor: removed unwanted spaces * refactor:removed unwanted spaces in validation.md * resolve conflict while rebasing * resolve conflict while rebasing * updated pytest script file * updated validation.md * addressed review comments * updated validation.md --------- Co-authored-by: takishida <38262981+takishida@users.noreply.github.com> * Removed gen1_switch_compatibility_check --------- Co-authored-by: GM <monrog2@gmail.com> Co-authored-by: Gabriel <gmonroy@cisco.com> Co-authored-by: tkishida <tkishida@cisco.com> Co-authored-by: takishida <38262981+takishida@users.noreply.github.com> Co-authored-by: psureshb <psureshb@cisco.com> Co-authored-by: Harinadh-Saladi <hsaladi@cisco.com> Co-authored-by: sanjanch <sanjanch@cisco.com> Co-authored-by: asraf-khan <anazar@cisco.com> Co-authored-by: Thatleft <detlef.sass@web.de> Co-authored-by: Detlef Sass <detlef.sass@de.bosch.com> Co-authored-by: veesenth_cisco <veesenth@cisco.com>

…age (#368) * Add arg to define thread limit - to throttle concurrent API calls when required (#355) * add `--max-threads` arg * fix bad descriptor errs/race conditions * update pytests * 353 the script incorrectly detects vpc and port channel interfaces as cscwh68103 invalid fabricpathep targets (#357) * specific testing for known failure conditions of cscwh68103 as to not catch valid scenarios * Update aci-preupgrade-validation-script.py mark version * print cleanup * Added code for CSCwo96334:/tmp directory disk space for snapshot storage * Updated fix version * Added clarification comment for /tmp snapshot handling * fix: Add cversion check for post_upgrade_cb_check (#377) * added validation for CFD CSCwp64296 (#307) Added rogue ep/coop exception mac check for the CFD CSCwp64296 * Added pre-upgrade validation for N9K-C9408 with more than 6 N9K-X9400-16W LEM's for the bug CSCws82819 (#354) * Added a new check for the bug 'CSCws82819N9K-C9408 boot loop on 16.1.2f and later with 6 or more LEMs' * New Validation for APIC Storage Inode Usage (F4388, F4389, F4390 equipment-full) (#361) * New Validation for APIC Storage Inode Usage (F4388, F4389, F4390 equipment-full) * Add new exception handling of invalid query filter in `icurl` due to, for example, a non-supported fault code on older versions * Add validation for multipod_modular_spine_bootscript_check - CSCwr66848 (#365) * Add check for CFD - CSCwr66848 * Update pytest.yml to run on vX.Y.Z branches * adding of cli parameters for user and password (#335) * added cli parameter for user (-u) and password (-p) to be used in an eased way for fully automated execution of 'aci-preupgrade-validation-script.py' * test: Add pytest --------- Co-authored-by: Detlef Sass <detlef.sass@de.bosch.com> * Corrected the typo for snapshot --------- Co-authored-by: GM <monrog2@gmail.com> Co-authored-by: Gabriel <gmonroy@cisco.com> Co-authored-by: tkishida <tkishida@cisco.com> Co-authored-by: takishida <38262981+takishida@users.noreply.github.com> Co-authored-by: psureshb <psureshb@cisco.com> Co-authored-by: sanjanch <sanjanch@cisco.com> Co-authored-by: asraf-khan <anazar@cisco.com> Co-authored-by: Thatleft <detlef.sass@web.de> Co-authored-by: Detlef Sass <detlef.sass@de.bosch.com>

dhaselva reviewed Feb 26, 2026

View reviewed changes

Comment thread aci-preupgrade-validation-script.py Outdated

dhaselva reviewed Feb 26, 2026

View reviewed changes

Comment thread aci-preupgrade-validation-script.py Outdated

dhaselva reviewed Feb 26, 2026

View reviewed changes

Comment thread aci-preupgrade-validation-script.py Outdated

dhaselva reviewed Feb 26, 2026

View reviewed changes

Comment thread docs/docs/validations.md Outdated

dhaselva reviewed Feb 26, 2026

View reviewed changes

Comment thread docs/docs/validations.md Outdated

dhaselva reviewed Feb 27, 2026

View reviewed changes

Comment thread aci-preupgrade-validation-script.py

dhaselva reviewed Mar 3, 2026

View reviewed changes

Comment thread docs/docs/validations.md

dhaselva reviewed Mar 3, 2026

View reviewed changes

Comment thread aci-preupgrade-validation-script.py Outdated

dhaselva reviewed Mar 3, 2026

View reviewed changes

Comment thread tests/checks/n9k_c9408_model_lem_count_check/test_n9k_c9408_model_lem_count_check.py Outdated

lovkeshsharma702 reviewed Mar 4, 2026

View reviewed changes

Comment thread docs/docs/validations.md Outdated

lovkeshsharma702 reviewed Mar 4, 2026

View reviewed changes

Comment thread aci-preupgrade-validation-script.py Outdated

lovkeshsharma702 reviewed Mar 4, 2026

View reviewed changes

Comment thread aci-preupgrade-validation-script.py Outdated

lovkeshsharma702 reviewed Mar 4, 2026

View reviewed changes

Comment thread aci-preupgrade-validation-script.py Outdated

muthu-ku reviewed Mar 9, 2026

View reviewed changes

Comment thread docs/docs/validations.md Outdated

muthu-ku reviewed Mar 9, 2026

View reviewed changes

Comment thread docs/docs/validations.md

Harinadh-Saladi added 5 commits March 9, 2026 12:08

Added a new check for the bug 'CSCws82819N9K-C9408 boot loop on 16.1.…

338af6c

…2f and later with 6 or more LEMs'

Removed if conditio for model check and modified the recommended actions

1712888

Updated recommended action message, method name and pytest file names

ae38859

Updated pytest file names in python file

7bc42c4

Updated the if condition for maintenance release,6.1(5e) and variable…

6f82308

… name for impacted nodes.Also, updated pytest test cases

lovkeshsharma702 reviewed Mar 10, 2026

View reviewed changes

Comment thread tests/checks/n9k_c9408_model_lem_count_check/test_n9k_c9408_model_lem_count_check.py

Comment thread aci-preupgrade-validation-script.py Outdated

Comment thread docs/docs/validations.md

Harinadh-Saladi commented Mar 10, 2026

View reviewed changes

Comment thread tests/checks/n9k_c9408_model_lem_count_check/test_n9k_c9408_model_lem_count_check.py

Addressed comments:recommended message,doc_url and rebased the code w…

1bea834

…ith latest changes

Harinadh-Saladi force-pushed the CSCws82819-c9408-boot-loop-check branch from b4e06c9 to 1bea834 Compare March 10, 2026 03:06

muthu-ku reviewed Mar 10, 2026

View reviewed changes

Comment thread aci-preupgrade-validation-script.py Outdated

Harinadh-Saladi added 3 commits March 10, 2026 06:28

Updated bugs section with the checks post rebase

5390298

Removed unwanted additional spaces

49aabe5

Removed duplicate check entries

31accd8

Harinadh-Saladi added 5 commits March 10, 2026 17:15

Update aci-preupgrade-validation-script.py

609d89f

Removed unwanted spaces

Update validations.md

029e735

Removed unwanted spaces

Updated validations.md file by removing unwanted space

f3f41ff

Updated validations.md file

fb8476b

Updated validations.md file by modifying the impact and recommendation message.

Updated validations.md

2483b35

Removed single quotes in the description.

lovkeshsharma702 reviewed Mar 11, 2026

View reviewed changes

Comment thread aci-preupgrade-validation-script.py Outdated

lovkeshsharma702 reviewed Mar 11, 2026

View reviewed changes

Comment thread docs/docs/validations.md

lovkeshsharma702 reviewed Mar 11, 2026

View reviewed changes

Harinadh-Saladi added 2 commits March 12, 2026 06:08

Updated aci-preupgrade-validation-script.py

fb7b313

Updated recommended action.

Updated validations.md

ef1282f

Updated workaround message.

lovkeshsharma702 approved these changes Mar 12, 2026

View reviewed changes

takishida added 3 commits April 2, 2026 23:54

Merge branch 'CSCws82819-c9408-boot-loop-check' of github.com:Harinad…

a936409

…h-Saladi/ACI-Pre-Upgrade-Validation-Script into CSCws82819-c9408-boot-loop-check

fix: Use NA only for version not applicable

c8a70a0

chore: Add tver missing handling

8c227d8

takishida changed the base branch from master to v4.1.0-dev April 3, 2026 07:35

takishida merged commit 6ccd6d9 into datacenter:v4.1.0-dev Apr 3, 2026
2 checks passed

Conversation

Harinadh-Saladi commented Feb 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dhaselva Mar 3, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Harinadh-Saladi commented Mar 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Harinadh-Saladi commented Mar 11, 2026

Uh oh!

Uh oh!

Uh oh!

lovkeshsharma702 left a comment

Choose a reason for hiding this comment

Uh oh!

takishida commented Apr 3, 2026

Example result outputs:

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Harinadh-Saladi commented Feb 25, 2026 •

edited

Loading

Harinadh-Saladi commented Mar 9, 2026 •

edited

Loading